.ipynb .pdf

repository open issue suggest edit

Contents

Movie-Watching Trajectories

Note

Last updated: 7 PM, 8/9/2020.

Note

This page will likely be split up in the future. It is currently organized like this for the convenience of working in one Jupyter Notebook. The split will also allow for better titles and headings.

from math import cos, sin, pi, sqrt
import random
import pickle as pkl
import numpy as np
import pandas as pd

import plotly.graph_objects as go
import plotly.express as px
from plotly.subplots import make_subplots
import plotly.offline as pyo
pyo.init_notebook_mode()

from IPython.display import display
from ipythonblocks import BlockGrid
from webcolors import name_to_rgb
from scipy import interpolate
import warnings
warnings.filterwarnings('ignore')

Trajectory Data

Movie-watching fMRI data were obtained from the Human Connectome Project. Participants were scanned while they watched movie excerpts and short clips, in which data were sampled every second. Principal component analysis (PCA) was used to reduce the trajectory data to three dimensions and saving it to a pandas dataframe in trajectories.pkl. Let’s import it and modify it for our needs.

We first need to calculate the mean trajectories for each clip, averaged across all participants. This will be stored in a new dataframe for just mean trajectories.

We also want to define where the mean trajectory ends, but just choosing one point would be too noisy. Trajectories eventually stagnate around a certain space for multiple time steps, so we can define this space as a sphere. We’ll define the sphere’s center point as the mean of a group of points near the end of the trajectory, where “end” is based on time. The radius will then be set as the farthest distance from the mean point to any individual point. We can then calculate the density of trajectory points inside the sphere. The goal is to find a sphere with the highest density. To accomplish this, the points included in the group is tested between the last 10 points to all points in the trajectory. The mean point and radius are then stored in the mean dataframe.

We can also store some information about each clip into a new dataframe. This info includes clip ID, clip name, and clip length.

The three dataframes can be stored in a .pkl file for easy importing without having to redo calculations. Let’s display the three dataframes we’ve created: traj_df, mean_df, and clipdata_df.

# import data
with open('trajectories.pkl', 'rb') as f:
    data = pkl.load(f)
traj_df = data['traj_df'] # pandas dataframe with 3D coordinates, time, particpant id (pid), clip id (clip) and  clip name as columns
clip_len = data['clip_len'] # array consisting number of time points indexed by clip_id

# add clip length to traj_df, then reorder traj_df
traj_df['clip_len'] = traj_df['clip'].transform(lambda x: clip_len[x])
traj_df = traj_df[['clip','clip_name','clip_len','pid','time','x','y','z']]

# create new df for each clip's mean trajectories
mean_df = traj_df.drop(columns=['pid']).groupby(['clip','clip_name','clip_len','time']).agg(np.mean).reset_index()

# calculate mean trajectory ends
end_x = np.array([])
end_y = np.array([])
end_z = np.array([])
end_r = np.array([])
for clip, group in mean_df.groupby('clip'):

    max_density = 0
    r = 0
    curr_clip_len = group['clip_len'].iloc[0]
    
    # iterate through number of points to include in end
    for num_points in range(10,curr_clip_len+1):
        
        # calculate mean of points
        temp_df = group.drop(columns=['clip','clip_name','clip_len','time']).iloc[curr_clip_len-num_points : curr_clip_len-1]
        mean = temp_df.agg(np.mean)
        
        # calculate min radius that includes all points
        max_r = -1
        for i,point in temp_df.iterrows():
            curr_r = sqrt((mean['x']-point['x'])**2 + (mean['y']-point['y'])**2 + (mean['z']-point['z'])**2)
            if (curr_r > max_r):
                max_r = curr_r
        
        # calculate greatest density of points in sphere
        num_points_in = 0
        for i,point in group.drop(columns=['clip','clip_name','clip_len','time']).iterrows():
            if (sqrt((mean['x']-point['x'])**2 + (mean['y']-point['y'])**2 + (mean['z']-point['z'])**2) <= max_r):
                num_points_in += 1

        curr_density = num_points_in**3 / (4/3*pi*max_r**3)
        if (curr_density > max_density):
            max_density = curr_density
            r = max_r
            best_end = mean
            
    # add end data to arrays
    end_x = np.concatenate((end_x, np.ones(curr_clip_len)*best_end['x']))
    end_y = np.concatenate((end_y, np.ones(curr_clip_len)*best_end['y']))
    end_z = np.concatenate((end_z, np.ones(curr_clip_len)*best_end['z']))
    end_r = np.concatenate((end_r, np.ones(curr_clip_len)*r))

mean_df['end_x'] = end_x
mean_df['end_y'] = end_y
mean_df['end_z'] = end_z
mean_df['end_r'] = end_r

# create new df for unique clip ids, names, and lengths
clipdata_df = pd.DataFrame({'clip':np.arange(0,len(clip_len)),
                            'clip_name':traj_df.clip_name.unique(),
                            'clip_len':clip_len},
                           columns=['clip','clip_name','clip_len'])

# save dataframes
with open("trajectories_updated.pkl", "wb") as f:
    pkl.dump({'traj_df':traj_df, 'mean_df':mean_df, 'clipdata_df':clipdata_df}, f)

#load
with open('trajectories_updated.pkl', 'rb') as f:
    data = pkl.load(f)
traj_df = data['traj_df']
mean_df = data['mean_df']
clipdata_df = data['clipdata_df']

# display dataframes
display(traj_df)
display(mean_df)
display(clipdata_df)

	clip	clip_name	clip_len	pid	time	x	y	z
0	0	testretest	84	1	0	-0.068375	0.292656	0.076036
1	0	testretest	84	1	1	-0.560828	0.290854	0.095379
2	0	testretest	84	1	2	0.248541	-0.024260	-0.019393
3	0	testretest	84	1	3	-0.021169	0.253559	1.618106
4	0	testretest	84	1	4	-0.218407	0.420255	2.193483
...	...	...	...	...	...	...	...	...
245399	14	starwars	256	76	251	-6.074567	-14.429848	13.255661
245400	14	starwars	256	76	252	-5.333982	-15.487228	15.741982
245401	14	starwars	256	76	253	-5.229411	-14.917367	16.472929
245402	14	starwars	256	76	254	-4.298551	-12.905822	15.564515
245403	14	starwars	256	76	255	-3.862932	-14.278425	16.305193

245404 rows × 8 columns

	clip	clip_name	clip_len	time	x	y	z	end_x	end_y	end_z	end_r
0	0	testretest	84	0	-0.535419	-0.584089	0.527666	-2.512531	-16.690886	14.976014	0.625368
1	0	testretest	84	1	-0.828858	-1.073595	0.652328	-2.512531	-16.690886	14.976014	0.625368
2	0	testretest	84	2	-0.948996	-1.365094	0.686838	-2.512531	-16.690886	14.976014	0.625368
3	0	testretest	84	3	-1.016553	-1.504698	0.718371	-2.512531	-16.690886	14.976014	0.625368
4	0	testretest	84	4	-1.068053	-1.757247	0.923804	-2.512531	-16.690886	14.976014	0.625368
...	...	...	...	...	...	...	...	...	...	...	...
2972	14	starwars	256	251	7.020762	-20.638204	2.571888	7.565658	-20.862614	2.771892	1.202724
2973	14	starwars	256	252	6.981013	-20.744694	2.663826	7.565658	-20.862614	2.771892	1.202724
2974	14	starwars	256	253	7.065092	-20.584518	2.731824	7.565658	-20.862614	2.771892	1.202724
2975	14	starwars	256	254	7.200529	-20.280288	2.620943	7.565658	-20.862614	2.771892	1.202724
2976	14	starwars	256	255	7.261687	-20.207359	2.578671	7.565658	-20.862614	2.771892	1.202724

2977 rows × 11 columns

	clip	clip_name	clip_len
0	0	testretest	84
1	1	twomen	245
2	2	bridgeville	222
3	3	pockets	189
4	4	overcome	65
5	5	inception	227
6	6	socialnet	260
7	7	oceans	250
8	8	flower	181
9	9	hotel	186
10	10	garden	205
11	11	dreary	143
12	12	homealone	233
13	13	brokovich	231
14	14	starwars	256

Let’s create a colorscale so we can easily visualize the 15 clips on one graph.

grid = BlockGrid(15,1,fill=(0,0,0))
grid.block_size = 50
grid.lines_on = False

colors = ['slategray','sienna','darkred','crimson','darkorange','darkgoldenrod','darkkhaki','mediumseagreen','darkgreen','darkcyan','cornflowerblue','mediumblue','blueviolet','purple','hotpink']
i = 0
for block in grid:
    color = name_to_rgb(colors[i])
    block.set_colors(color[0],color[1],color[2])
    i+=1
    
grid.show()

Let’s plot the mean trajectories and their ends.

Note

All plots on this page are interactive. You can zoom by scrolling, rotate by clicking and dragging, and get info on hover. You can also use the modebar on the upper left to zoom, rotate, pan, or reset camera.

plotly_data = []

for clip, clip_name in enumerate(clipdata_df['clip_name']):

    # mean trajectories
    temp_df = mean_df[(mean_df.clip_name==clip_name)]
    custom_df = temp_df[['time','clip_len']]
    custom_df['clip_len'] = custom_df['clip_len']-1
    mean_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=custom_df,
        mode='markers+lines',
        marker={'size':2, 'color': colors[clip]},
        line={'width':4, 'color': colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}')
    plotly_data.append(mean_traj)
    
    # end area
    theta = np.linspace(0,2*pi,50)
    phi = np.linspace(0,pi,50)
    r = temp_df['end_r'].iloc[0]
    x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
    y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
    z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
    #custom = np.repeat([temp_df[['end_x','end_y','end_z','end_r']].head(50).to_numpy().T], repeats=50, axis=0)
    #custom = np.array([temp_df[['end_x','end_y','end_z','end_r']].head(50).to_numpy().T] * 50)
    #custom = np.repeat(np.expand_dims(temp_df[['end_x','end_y','end_z','end_r']].head(50).to_numpy().T, axis=1), repeats=50, axis=1)
    sphere = go.Surface(
        x=x,
        y=y,
        z=z,
        customdata=custom,
        #customdata=[temp_df['end_x'].iloc[0], temp_df['end_y'].iloc[0], temp_df['end_z'].iloc[0], temp_df['end_r'].iloc[0]],
        opacity=0.3,
        name=clip_name,
        legendgroup=clip_name,
        hoverinfo='skip',
        #hovertemplate='End x: %{customdata[0]:.3f}<br>End y: %{customdata[1]:.3f}<br>End z: %{customdata[2]:.3f}<br>End r: %{customdata[3]:.3f}',
        showscale=False,
        colorscale=[colors[clip],colors[clip]])
    plotly_data.append(sphere)

# formatting
plotly_layout = go.Layout(
    showlegend=True, 
    autosize=False,
    width=800, 
    height=800,
    margin={'l':0, 'r':0, 't':40, 'b':90},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'x':0.5,
            'y':-0.08,
            'tracegroupgap':2},
    title={'text':'Mean Trajectories',
           'xanchor':'center',
           'yanchor':'top',
           'x':0.5,
           'y':0.98},
    annotations=[{'xref':'paper',
                  'yref':'paper',
                  'xanchor':'center',
                  'yanchor':'bottom',
                  'x':0.5,
                  'y':-0.13,
                  'showarrow':False,
                  'text':'<b>Fig. 1.</b> Mean trajectory for each clip, averaged over all participants. Each trajectory\'s end is represented as a sphere.'}],
    updatemenus=[{'type':'buttons',
                  'direction':'left',
                  'pad':{'l':0, 'r':0, 't':0, 'b':0},
                  'xanchor':'left',
                  'yanchor':'top',
                  'x':0,
                  'y':1.055,
                  'buttons':[
                      {'label':'Show Ends',
                       'method': 'update',
                       'args':[{'visible': [True]*clipdata_df.shape[0]*2}]},
                      {'label':'Hide Ends',
                       'method': 'update',
                       'args':[{'visible': [True,False]*clipdata_df.shape[0]}]}
                  ]}])

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation','hoverClosest3d']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

Individual vs Mean Trajectories

Mean trajectories are not entirely representative of our data. Let’s look at two clips that have similar mean trajectories, “ocean” and “brokovich” in this case. We’ll plot both their mean and individual trajectories (represented as individual unconnected points).

plotly_data = []



# OCEANS

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='oceans')]
temp_df['clip_len'] = temp_df['clip_len']-1
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'mediumblue'},
    line={'width':4, 'color':'mediumblue'},
    name='oceans',
    legendgroup='oceans',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['blue','blue'])
plotly_data.append(sphere)

## individual trajectories
temp_df = traj_df[(traj_df.clip_name=='oceans')]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df['time'],
    mode='markers',
    marker={'size':0.5, 'color':'mediumblue'},
    opacity=0.5,
    name='oceans',
    legendgroup='oceans',
    showlegend=False,
    hoverinfo='skip')
plotly_data.append(pid_traj)



# BROKOVICH

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='brokovich')]
temp_df['clip_len'] = temp_df['clip_len']-1
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'crimson'},
    line={'width':4, 'color':'crimson'},
    name='brokovich',
    legendgroup='brokovich',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['red','red'])
plotly_data.append(sphere)

## individual trajectories
temp_df = traj_df[(traj_df.clip_name=='brokovich')]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df['time'],
    mode='markers',
    marker={'size':0.5, 'color':'crimson'},
    opacity=0.5,
    name='brokovich',
    legendgroup='brokovich',
    showlegend=False,
    hoverinfo='skip')
plotly_data.append(pid_traj)



# formatting
plotly_layout = go.Layout(
    showlegend=True, 
    autosize=False,
    width=800, 
    height=600,
    margin={'l':0, 'r':0, 't':35, 'b':70},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'y':-0.06,
            'x':0.5},
    title={'text':'Mean and Individual Trajectories',
           'xanchor':'center',
           'yanchor':'top',
           'x':0.5,
           'y':0.98},
    annotations=[{'xref':'paper',
                  'yref':'paper',
                  'xanchor':'center',
                  'yanchor':'bottom',
                  'x':0.5,
                  'y':-0.15,
                  'showarrow':False,
                  'text':'<b>Fig. 2.</b> Mean trajectories (represented as points connected by lines) and individual trajectories<br>(represented as individual unconnected points) for the clips "oceans" and "brokovich".'}])

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation','hoverClosest3d']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

We see that the majority of individual points lie in clouds near the end of their corresponding mean trajectories, with many other points surrounding the mean trajectories. This behavior indicates that most individual trajectories follow a similar path and end in a similar location, especially since the mean trajectories themselves stay in a relatively close clump for the latter half of the clips.

However, there are a fair amount of points that don’t follow this trend. Let’s plot the endpoints of all individual trajectories to see where they finish at the end of the clip.

plotly_data = []



# OCEANS

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='oceans')]
temp_df['clip_len'] = temp_df['clip_len']-1
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'mediumblue'},
    line={'width':4, 'color':'mediumblue'},
    name='oceans',
    legendgroup='oceans',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['mediumblue','mediumblue'])
plotly_data.append(sphere)



## individual trajectories
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.time==clipdata_df[clipdata_df.clip_name=='oceans']['clip_len'].iloc[0]-1)]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','pid']],
    mode='markers',
    marker={'size':4, 'color':'mediumblue'},
    opacity=0.5,
    name='oceans',
    legendgroup='oceans',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
plotly_data.append(pid_traj)



# # BROKOVICH

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='brokovich')]
temp_df['clip_len'] = temp_df['clip_len']-1
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'crimson'},
    line={'width':4, 'color':'crimson'},
    name='brokovich',
    legendgroup='brokovich',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['crimson','crimson'])
plotly_data.append(sphere)

## individual trajectories
temp_df = traj_df[(traj_df.clip_name=='brokovich') & (traj_df.time==clipdata_df[clipdata_df.clip_name=='brokovich']['clip_len'].iloc[0]-1)]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','pid']],
    mode='markers',
    marker={'size':4, 'color':'crimson'},
    opacity=0.5,
    name='brokovich',
    legendgroup='brokovich',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
plotly_data.append(pid_traj)



# formatting
plotly_layout = go.Layout(
    showlegend=True, 
    autosize=False,
    width=800, 
    height=600,
    margin={'l':0, 'r':0, 't':35, 'b':60},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'y':-0.06,
            'x':0.5},
    title={'text':'Individual Trajectory Endpoints',
           'xanchor':'center',
           'yanchor':'top',
           'x':0.5,
           'y':0.98},
    annotations=[{'xref':'paper',
                  'yref':'paper',
                  'xanchor':'center',
                  'yanchor':'bottom',
                  'x':0.5,
                  'y':-0.12,
                  'showarrow':False,
                  'text':'<b>Fig. 3.</b> Mean trajectories and individual trajectory endpoints (last time step) for the clips "oceans" and "brokovich".'}])

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation','hoverClosest3d']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

It appears that most trajectories end at a similar location as the mean trajectories. There aren’t any other noticeable groups, so let’s take a look at some of the trajectories that fall into the cloud around the mean trajectories.

Note

Individual trajectories shown below have not yet been smoothed. Figures in this section are not numbered or captioned yet.

plotly_data = []



# OCEANS

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='oceans')]
temp_df['clip_len'] = temp_df['clip_len']-1
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'mediumblue'},
    line={'width':4, 'color':'mediumblue'},
    name='oceans',
    legendgroup='oceans',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['mediumblue','mediumblue'])
plotly_data.append(sphere)

## individual trajectories
for pid in ([32,57,73]):
    temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
    temp_df['clip_len'] = temp_df['clip_len']-1
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','clip_len','pid']],
        mode='markers+lines',
        marker={'size':1, 'color':'mediumblue'},
        opacity=0.5,
        name='oceans',
        legendgroup='oceans',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
    plotly_data.append(pid_traj)
    
    temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid) & (traj_df.time==clipdata_df[clipdata_df.clip_name=='oceans']['clip_len'].iloc[0]-1)]
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','pid']],
        mode='markers',
        marker={'size':8, 'color':'mediumblue'},
        opacity=0.7,
        name='oceans',
        legendgroup='oceans',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
    plotly_data.append(pid_traj)



# BROKOVICH

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='brokovich')]
temp_df['clip_len'] = temp_df['clip_len']-1
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'crimson'},
    line={'width':4, 'color':'crimson'},
    name='brokovich',
    legendgroup='brokovich',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['crimson','crimson'])
plotly_data.append(sphere)

## individual trajectories
for pid in ([2,20,72]):
    temp_df = traj_df[(traj_df.clip_name=='brokovich') & (traj_df.pid==pid)]
    temp_df['clip_len'] = temp_df['clip_len']-1
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','clip_len','pid']],
        mode='markers+lines',
        marker={'size':1, 'color':'crimson'},
        opacity=0.5,
        name='brokovich',
        legendgroup='brokovich',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
    plotly_data.append(pid_traj)
    
    temp_df = traj_df[(traj_df.clip_name=='brokovich') & (traj_df.pid==pid) & (traj_df.time==clipdata_df[clipdata_df.clip_name=='brokovich']['clip_len'].iloc[0]-1)]
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','pid']],
        mode='markers',
        marker={'size':8, 'color':'crimson'},
        opacity=0.7,
        name='brokovich',
        legendgroup='brokovich',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
    plotly_data.append(pid_traj)
    



# formatting
plotly_layout = go.Layout(showlegend=True, 
                          autosize=False,
                          width=800, 
                          height=600,
                          margin={'l':0, 'r':0, 't':35, 'b':0},
                          legend={'orientation':'h',
                                  'itemsizing':'constant',
                                  'xanchor':'center',
                                  'yanchor':'bottom',
                                  'y':-0.055,
                                  'x':0.5},
                          title={'text':'Mean-Like Trajectories for \"oceans\" and \"brokovich\"',
                                 'xanchor':'center',
                                 'yanchor':'top',
                                 'x':0.5,
                                 'y':0.98})

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation','hoverClosest3d']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

Just from looking at a few individual trajectories, we can notice that some trajectories do meander a fair amount before ending at the mean trajectory cloud. Notable examples include oceans 32, oceans 57, brokovich 20, and brokovich 72. Some other trajectories such as oceans 73 and brokovich 2 do follow a more similar path to the mean trajectory, but they still exhibit slight meandering.

We should also take a look at some trajectories that ended at a completely different point from the mean trajectories.

plotly_data = []



# OCEANS

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='oceans')]
temp_df['clip_len'] = temp_df['clip_len']-1
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'mediumblue'},
    line={'width':4, 'color':'mediumblue'},
    name='oceans',
    legendgroup='oceans',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['mediumblue','mediumblue'])
plotly_data.append(sphere)

## individual trajectories
for pid in ([17,18]):
    temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
    temp_df['clip_len'] = temp_df['clip_len']-1
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','clip_len','pid']],
        mode='markers+lines',
        marker={'size':1, 'color':'mediumblue'},
        opacity=0.5,
        name='oceans',
        legendgroup='oceans',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
    plotly_data.append(pid_traj)
    
    temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid) & (traj_df.time==clipdata_df[clipdata_df.clip_name=='oceans']['clip_len'].iloc[0]-1)]
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','pid']],
        mode='markers',
        marker={'size':8, 'color':'mediumblue'},
        opacity=0.5,
        name='oceans',
        legendgroup='oceans',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
    plotly_data.append(pid_traj)



# BROKOVICH

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='brokovich')]
temp_df['clip_len'] = temp_df['clip_len']-1
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'crimson'},
    line={'width':4, 'color':'crimson'},
    name='brokovich',
    legendgroup='brokovich',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['crimson','crimson'])
plotly_data.append(sphere)

## individual trajectories
for pid in ([49,56]):
    temp_df = traj_df[(traj_df.clip_name=='brokovich') & (traj_df.pid==pid)]
    temp_df['clip_len'] = temp_df['clip_len']-1
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','clip_len','pid']],
        mode='markers+lines',
        marker={'size':1, 'color':'crimson'},
        opacity=0.5,
        name='brokovich',
        legendgroup='brokovich',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
    plotly_data.append(pid_traj)
    
    temp_df = traj_df[(traj_df.clip_name=='brokovich') & (traj_df.pid==pid) & (traj_df.time==clipdata_df[clipdata_df.clip_name=='brokovich']['clip_len'].iloc[0]-1)]
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','pid']],
        mode='markers',
        marker={'size':8, 'color':'crimson'},
        opacity=0.5,
        name='brokovich',
        legendgroup='brokovich',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
    plotly_data.append(pid_traj)
    



# formatting
plotly_layout = go.Layout(showlegend=True, 
                          autosize=False,
                          width=800, 
                          height=600,
                          margin={'l':0, 'r':0, 't':35, 'b':0},
                          legend={'orientation':'h',
                                  'itemsizing':'constant',
                                  'xanchor':'center',
                                  'yanchor':'bottom',
                                  'y':-0.055,
                                  'x':0.5},
                          title={'text':'Mean-Unlike Trajectories for \"oceans\" and \"brokovich\"',
                                 'xanchor':'center',
                                 'yanchor':'top',
                                 'x':0.5,
                                 'y':0.98})

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation','hoverClosest3d']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

These trajectories are completely different from the mean trajectories. They begin at an entirely different direction and continue to travel in a fashion that is seemingly unrelated to their respective mean trajectories. However, it is important to remember that these mean-unlike trajectories are few in quantity in comparison to mean-unlike trajectories.

Note

It may be useful to look into calculating mean trajectories without mean-unlike trajectories, which act like outliers. This may allow us to quantify how many trajetories are truly mean-like by choosing the individual trajectories that end within a certain distance of the mean trajectory’s end.

Smoothing

The individual trajectories, as show previously, are much more noisy than mean trajectories. To reduce noise and allow for better visualization, we’ll first smooth individual trajectories. This can be done many different ways, but we’ll just look at a couple methods.

The simplest smoothing technique is a rolling mean, which finds the mean value within a window that includes the current and some previous points. In this example, we’ll use a window size of 5. This means that each point in the smoothed trajectory will be the mean of the current point and 4 previous points in the original trajectory. Note that this leaves out the first four time points since there aren’t enough points in the window, but this doesn’t have a big impact on the final results.

Another smoothing method is splines, functions defined piecewise by polynomials that meet at points \(t_i\) known as knots. A spline function with degree \(k\) guarantees that the first \(k-1\) derivatives are continuous. We want to be able to calculate first and second derivatives of our trajectories, so we will use a spline function of degree 3. To find these piecewise polynomials, we can use basis splines (B-splines), which are calculated with the Cox–de Boor recursion formula. By representing the spline function with B-splines, we reduce the dimension from the original input space to the space spanned by the basis functions, guaranteeing that all spline functions can be uniquely built from a linear combination of B-splines. Therefore, simply providing coefficients for each B-spline is sufficient information to generate a spline function.

fig_traj = make_subplots(rows=3, cols=1, 
                         vertical_spacing=0.05,
                         subplot_titles=('Unsmoothed', 'Smoothed (Rolling Mean)', 'Smoothed (Splines)'), 
                         specs=[[{'type':'scatter3d'}], [{'type':'scatter3d'}], [{'type':'scatter3d'}]])

pid = 57
    t
# unsmoothed
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
temp_df['clip_len'] = temp_df['clip_len']-1
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len','pid']],
    mode='markers+lines',
    line={'width':4, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    opacity=0.5,
    name='Unsmoothed',
    legendgroup='Unsmoothed',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
fig_traj.add_trace(pid_traj, row=1, col=1)

# smoothed (rolling mean)
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
temp_df['clip_len'] = temp_df['clip_len']-1
temp_df[['x','y','z']] = temp_df[['x','y','z']].rolling(window=5).mean()
temp_df = temp_df.drop(temp_df.index[[0,1,2,3]])
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len','pid']],
    mode='markers+lines',
    line={'width':4, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    opacity=0.5,
    name='Smoothed (Rolling Mean)',
    legendgroup='Smoothed (Rolling Mean)',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
fig_traj.add_trace(pid_traj, row=2, col=1)

# smoothed (splines)
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
temp_df['clip_len'] = temp_df['clip_len']-1
data = temp_df[['x','y','z']].to_numpy()
tck, u = interpolate.splprep(data.T, k=3)
data = interpolate.splev(np.linspace(0,1,temp_df['clip_len'].iloc[0]+1), tck, der=0)
temp_df['x'] = data[0]
temp_df['y'] = data[1]
temp_df['z'] = data[2]
# data = interpolate.splev(np.linspace(0,1,clip_len*10), tck, der=0)
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len','pid']],
    # customdata=np.vstack([np.linspace(0,clip_len-1,clip_len*10), np.ones(clip_len*10)*(clip_len-1), np.ones(clip_len*10)*pid]).T,
    mode='markers+lines',
    line={'width':4, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    opacity=0.5,
    name='Smoothed (Splines)',
    legendgroup='Smoothed (Splines)',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}'
)
fig_traj.add_trace(pid_traj, row=3, col=1)
    
# formatting
fig_traj.update_layout(
    autosize=False,
    width=800, 
    height=1000,
    margin={'l':0, 'r':0, 't':70, 'b':60},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'y':-0.055,
            'x':0.5},
    title={'text':'Smoothing Individual Trajectories',
            'xanchor':'center',
            'yanchor':'top',
            'x':0.5,
            'y':0.98})
fig_traj['layout']['annotations'] += (
    {'xref':'paper',
     'yref':'paper',
     'xanchor':'center',
     'yanchor':'bottom',
     'x':0.5,
     'y':-0.07,
     'showarrow':False,
     'text':'<b>Fig. 4.</b> Individual trajectories for "oceans" participant 57. (A) Original unsmoothed trajectory.<br>(B) Smoothed with rolling mean using a window size of 5. (C) Smoothed with cubic splines.'
    },
)

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation','hoverClosest3d']}

# sync camera (only available in Jupyter, does not appear in Jupyter-Book)
# fig = go.FigureWidget(fig_traj)
# def cam_change(layout, camera):
#     fig.layout.scene1.camera = camera
#     fig.layout.scene3.camera = camera
# fig.layout.scene2.on_change(cam_change, 'camera')
# fig

fig_traj.show(config=plotly_config)

Splines seem to be the most effective at smoothing. However, it is important to point out that smoothing has a tradeoff. Although more smoothing creates a trajectory with less noise, it also strays from the unsmoothed trajectory which may cause the smoothed trajectory to lose some of the original information. We can more clearly see this diffence between unsmoothed and smooth trajectories by plotting both trajectories on one plot.

Note

Click on the icons in the legend to choose which trajectories to see. A single click toggles the clicked trajectory from the plot. A double click toggles all other trajectories.

plotly_data = []

pid = 57
    
# unsmoothed
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
temp_df['clip_len'] = temp_df['clip_len']-1
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len','pid']],
    mode='markers+lines',
    line={'width':4, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    opacity=0.3,
    name='Unsmoothed',
    legendgroup='Unsmoothed',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
plotly_data.append(pid_traj)

# smoothed (splines)
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
temp_df['clip_len'] = temp_df['clip_len']-1
data = temp_df[['x','y','z']].to_numpy()
tck, u = interpolate.splprep(data.T, k=3)
data = interpolate.splev(np.linspace(0,1,temp_df['clip_len'].iloc[0]+1), tck, der=0)
temp_df['x'] = data[0]
temp_df['y'] = data[1]
temp_df['z'] = data[2]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    mode='markers+lines',
    customdata=temp_df[['time','clip_len','pid']],
    line={'width':5, 'color':'crimson'},
    marker={'size':1, 'color':'crimson'},
    opacity=0.5,
    name='Smoothed',
    legendgroup='Smoothed',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}'
)
plotly_data.append(pid_traj)

# legend
pid_traj = go.Scatter3d(
    x=[None], y=[None], z=[None],
    mode='markers+lines',
    line={'width':4, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    name='Unsmoothed',
    legendgroup='Unsmoothed',
    showlegend=True)
plotly_data.append(pid_traj)

pid_traj = go.Scatter3d(
    x=[None], y=[None], z=[None],
    mode='markers+lines',
    line={'width':5, 'color':'crimson'},
    marker={'size':1, 'color':'crimson'},
    name='Smoothed (Splines)',
    legendgroup='Smoothed (Splines)',
    showlegend=True)
plotly_data.append(pid_traj)

# formatting
plotly_layout = go.Layout(
    autosize=False,
    width=800, 
    height=600,
    margin={'l':0, 'r':0, 't':35, 'b':70},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'y':-0.065,
            'x':0.5},
    title={'text':'Smoothing Individual Trajectories',
            'xanchor':'center',
            'yanchor':'top',
            'x':0.5,
            'y':0.98},
    annotations=[{'xref':'paper',
                  'yref':'paper',
                  'xanchor':'center',
                  'yanchor':'bottom',
                  'x':0.5,
                  'y':-0.15,
                  'showarrow':False,
                  'text':'<b>Fig. 5.</b> Individual trajectories for "oceans" participant 57.<br>(Blue) Original unsmoothed trajectory. (Red) Smoothed with cubic splines.'}])
                  
plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation','hoverClosest3d']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

Quantifying “Bending”

From looking at individual trajectories, we can notice that they all have a significant amount of “bending”. We can define some ways to quantify this “bending” to see if we can find anything interesting.

Derivatives

We can think of a second derivative as acceleration, where higher acceleration indicates that the trajectory is changing direction. However, our trajectory data is discrete, so we need to modify the definition of a derivative to fit our needs. Given \(\Delta t = 1\), we’ll use the following definition:

\[\frac{d}{dt}\mathbf{r}(t) \approx \frac{\mathbf{r}(t)-\mathbf{r}(t-\Delta t)}{\Delta t} = \mathbf{r}(t)-\mathbf{r}(t-1)\]

Note that higher degree derivatives are simply defined using the first derivative. For example, the second derivative is defined as:

\[\frac{d^2}{dt^2}\mathbf{r}(t) = \frac{d}{dt}\left(\frac{d}{dt}\mathbf{r}(t)\right)\]

fig_traj = make_subplots(rows=2, cols=2, 
                         vertical_spacing=0.05, horizontal_spacing=0.05,
                         subplot_titles=('First Derivative', 'Second Derivative'), 
                         specs=[[{'type':'scatter3d'}, {'type':'scatter3d'}], [{'type':'scatter3d'}, {'type':'scatter3d'}]])
pid = 57
    
# unsmoothed
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
temp_df['clip_len'] = temp_df['clip_len']-1
temp_df[['x_der','y_der','z_der']] = temp_df['time'].transform(lambda x: temp_df[['x','y','z']].iloc[x]-temp_df[['x','y','z']].iloc[x-1])
temp_df[['x_derr','y_derr','z_derr']] = temp_df['time'].transform(lambda x: temp_df[['x_der','y_der','z_der']].iloc[x]-temp_df[['x_der','y_der','z_der']].iloc[x-1])
temp_df = temp_df.iloc[ -(temp_df['clip_len'].iloc[0]-1): ]

pid_traj = go.Scatter3d(
    x=temp_df['x_der'],
    y=temp_df['y_der'],
    z=temp_df['z_der'],
    customdata=temp_df[['time','clip_len','pid']],
    mode='markers+lines',
    line={'width':4, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    opacity=0.5,
    name='Unsmoothed',
    legendgroup='Unsmoothed',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
fig_traj.add_trace(pid_traj, row=1, col=1)

pid_traj = go.Scatter3d(
    x=temp_df['x_derr'],
    y=temp_df['y_derr'],
    z=temp_df['z_derr'],
    customdata=temp_df[['time','clip_len','pid']],
    mode='markers+lines',
    line={'width':4, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    opacity=0.5,
    name='Unsmoothed',
    legendgroup='Unsmoothed',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
fig_traj.add_trace(pid_traj, row=1, col=2)

# smoothed (splines)
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
data = temp_df[['x','y','z']].to_numpy()
tck, u = interpolate.splprep(data.T, k=3)
data = interpolate.splev(np.linspace(0,1,temp_df['clip_len'].iloc[0]), tck, der=0)
temp_df['x'] = data[0]
temp_df['y'] = data[1]
temp_df['z'] = data[2]
temp_df[['x_der','y_der','z_der']] = temp_df['time'].transform(lambda x: temp_df[['x','y','z']].iloc[x]-temp_df[['x','y','z']].iloc[x-1])
temp_df[['x_derr','y_derr','z_derr']] = temp_df['time'].transform(lambda x: temp_df[['x_der','y_der','z_der']].iloc[x]-temp_df[['x_der','y_der','z_der']].iloc[x-1])
#temp_df[['x_derr','y_derr','z_derr']] = temp_df['time'].transform(lambda x: temp_df[['x','y','z']].iloc[x]-2*temp_df[['x','y','z']].iloc[x-1]+temp_df[['x','y','z']].iloc[x-2])
temp_df = temp_df.iloc[ -(temp_df['clip_len'].iloc[0]-2): ]
temp_df['clip_len'] = temp_df['clip_len']-1

pid_traj = go.Scatter3d(
    x=temp_df['x_der'],
    y=temp_df['y_der'],
    z=temp_df['z_der'],
    customdata=temp_df[['time','clip_len','pid']],
    mode='markers+lines',
    line={'width':4, 'color':'crimson'},
    marker={'size':1, 'color':'crimson'},
    opacity=0.5,
    name='Smoothed (Splines)',
    legendgroup='Smoothed (Splines)',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
fig_traj.add_trace(pid_traj, row=2, col=1)

pid_traj = go.Scatter3d(
    x=temp_df['x_derr'],
    y=temp_df['y_derr'],
    z=temp_df['z_derr'],
    customdata=temp_df[['time','clip_len','pid']],
    mode='markers+lines',
    line={'width':4, 'color':'crimson'},
    marker={'size':1, 'color':'crimson'},
    opacity=0.5,
    name='Smoothed',
    legendgroup='Smoothed',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
fig_traj.add_trace(pid_traj, row=2, col=2)
    
# legend
pid_traj = go.Scatter3d(
    x=[None], y=[None], z=[None],
    mode='markers+lines',
    line={'width':4, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    name='Unsmoothed',
    legendgroup='Unsmoothed',
    showlegend=True)
fig_traj.add_trace(pid_traj, row=1, col=1)

pid_traj = go.Scatter3d(
    x=[None], y=[None], z=[None],
    mode='markers+lines',
    line={'width':6, 'color':'crimson'},
    marker={'size':1, 'color':'crimson'},
    name='Smoothed',
    legendgroup='Smoothed',
    showlegend=True)
fig_traj.add_trace(pid_traj, row=1, col=1)
    
# formatting
fig_traj.update_layout(
    autosize=False,
    width=800, 
    height=800, 
    margin={'l':0, 'r':0, 't':70, 'b':80},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'y':-0.05,
            'x':0.5},
    title={'text':'Individual Trajectory Derivatives (Vector)',
            'xanchor':'center',
            'yanchor':'top',
            'x':0.5,
            'y':0.98})
fig_traj['layout']['annotations'] += (
    {'xref':'paper',
     'yref':'paper',
     'xanchor':'center',
     'yanchor':'bottom',
     'x':0.5,
     'y':-0.12,
     'showarrow':False,
     'text':'<b>Fig. 6.</b> First and second derivatives (in vector form) of individual trajectories for "oceans" participant 57.<br>(Blue) Original unsmoothed trajectory. (Red) Smoothed with cubic splines.'
    },
)

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation','hoverClosest3d']}

fig_traj.show(config=plotly_config)

Leaving the derivatives in vector form doesn’t really tell us anything because we can’t visualize it well. Instead, we can try taking the magnitude of the derivative to get a scalar value, which we can then plot in two dimensions. Given \(\Delta t = 1\), this yields

\[\bigg\rvert\bigg\rvert\ \frac{d}{dt}\mathbf{r}(t)\ \bigg\rvert\bigg\rvert \approx \bigg\rvert\bigg\rvert\ \frac{\mathbf{r}(t)-\mathbf{r}(t-\Delta t)}{\Delta t}\ \bigg\rvert\bigg\rvert = \bigg\rvert\bigg\rvert\ \mathbf{r}(t)-\mathbf{r}(t-1)\ \bigg\rvert\bigg\rvert\]

fig_traj = make_subplots(rows=2, cols=1, 
                         shared_xaxes=True,
                         vertical_spacing=0.07,
                         subplot_titles=('First Derivative', 'Second Derivative'), 
                         specs=[[{'type':'scatter'}], [{'type':'scatter'}]])
pid = 57

# unsmoothed
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
temp_df[['x_der','y_der','z_der']] = temp_df['time'].transform(lambda x: temp_df[['x','y','z']].iloc[x]-temp_df[['x','y','z']].iloc[x-1])
temp_df['der'] = temp_df['time'].transform(lambda x: sqrt(temp_df['x_der'].iloc[x]**2 + temp_df['y_der'].iloc[x]**2 + temp_df['z_der'].iloc[x]**2))
temp_df[['x_derr','y_derr','z_derr']] = temp_df['time'].transform(lambda x: temp_df[['x_der','y_der','z_der']].iloc[x]-temp_df[['x_der','y_der','z_der']].iloc[x-1])
temp_df['derr'] = temp_df['time'].transform(lambda x: sqrt(temp_df['x_derr'].iloc[x]**2 + temp_df['y_derr'].iloc[x]**2 + temp_df['z_derr'].iloc[x]**2))
temp_df = temp_df.iloc[ -(temp_df['clip_len'].iloc[0]-2): ]
temp_df['clip_len'] = temp_df['clip_len']-1

pid_traj = go.Scatter(
    x=temp_df['time'],
    y=temp_df['der'],
    customdata=temp_df[['clip_len','pid']],
    mode='markers+lines',
    line={'width':2, 'color':'mediumblue'},
    marker={'size':4, 'color':'mediumblue'},
    name='Unsmoothed',
    legendgroup='Unsmoothed',
    showlegend=True,
    hovertemplate='1st der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>pid: %{customdata[1]}')
fig_traj.add_trace(pid_traj, row=1, col=1)

pid_traj = go.Scatter(
    x=temp_df['time'],
    y=temp_df['derr'],
    customdata=temp_df[['clip_len','pid']],
    mode='markers+lines',
    line={'width':2, 'color':'mediumblue'},
    marker={'size':4, 'color':'mediumblue'},
    name='Unsmoothed',
    legendgroup='Unsmoothed',
    showlegend=False,
    hovertemplate='2nd der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>pid: %{customdata[1]}')
fig_traj.add_trace(pid_traj, row=2, col=1)

# smoothed (splines)
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
temp_df['clip_len'] = temp_df['clip_len']-1
data = temp_df[['x','y','z']].to_numpy()
tck, u = interpolate.splprep(data.T, k=3)
data = interpolate.splev(np.linspace(0,1,temp_df['clip_len'].iloc[0]+1), tck, der=0)
temp_df['x'] = data[0]
temp_df['y'] = data[1]
temp_df['z'] = data[2]
temp_df[['x_der','y_der','z_der']] = temp_df['time'].transform(lambda x: temp_df[['x','y','z']].iloc[x]-temp_df[['x','y','z']].iloc[x-1])
temp_df['der'] = temp_df['time'].transform(lambda x: sqrt(temp_df['x_der'].iloc[x]**2 + temp_df['y_der'].iloc[x]**2 + temp_df['z_der'].iloc[x]**2))
temp_df[['x_derr','y_derr','z_derr']] = temp_df['time'].transform(lambda x: temp_df[['x','y','z']].iloc[x]-2*temp_df[['x','y','z']].iloc[x-1]+temp_df[['x','y','z']].iloc[x-2])
temp_df['derr'] = temp_df['time'].transform(lambda x: sqrt(temp_df['x_derr'].iloc[x]**2 + temp_df['y_derr'].iloc[x]**2 + temp_df['z_derr'].iloc[x]**2))
temp_df = temp_df.iloc[ -(temp_df['clip_len'].iloc[0]-1): ]

pid_traj = go.Scatter(
    x=temp_df['time'],
    y=temp_df['der'],
    customdata=temp_df[['clip_len','pid']],
    mode='markers+lines',
    line={'width':2, 'color':'crimson'},
    marker={'size':4, 'color':'crimson'},
    name='Smoothed',
    legendgroup='Smoothed',
    showlegend=True,
    hovertemplate='1st der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>pid: %{customdata[1]}'
)
fig_traj.add_trace(pid_traj, row=1, col=1)

pid_traj = go.Scatter(
    x=temp_df['time'],
    y=temp_df['derr'],
    customdata=temp_df[['clip_len','pid']],
    mode='markers+lines',
    line={'width':2, 'color':'crimson'},
    marker={'size':4, 'color':'crimson'},
    name='Smoothed',
    legendgroup='Smoothed',
    showlegend=False,
    hovertemplate='2nd der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>pid: %{customdata[1]}'
)
fig_traj.add_trace(pid_traj, row=2, col=1)

# formatting
fig_traj.update_layout(
    autosize=False,
    width=800, 
    height=800, 
    margin={'l':0, 'r':0, 't':70, 'b':100},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'y':-0.07,
            'x':0.5},
    title={'text':'Individual Trajectory Derivatives (Scalar)',
            'xanchor':'center',
            'yanchor':'top',
            'x':0.5,
            'y':0.98},
    hovermode='x')
fig_traj['layout']['annotations'] += (
    {'xref':'paper',
     'yref':'paper',
     'xanchor':'center',
     'yanchor':'bottom',
     'x':0.5,
     'y':-0.15,
     'showarrow':False,
     'text':'<b>Fig. 7.</b> First and second derivatives (in scalar form) of individual trajectories for "oceans" participant 57.<br>(Blue) Original unsmoothed trajectory. (Red) Smoothed with cubic splines.'
    },
)

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['autoScale2d','toggleSpikelines','hoverClosestCartesian','hoverCompareCartesian','lasso2d','select2d']}

fig_traj.show(config=plotly_config)

The smoothed trajectory has smaller first and second derivatives than the unsmoothed trajectory. This suggests that the trajectory is indeed reducing noise, making the trajectory take “smoother” paths with less drastic change.

Let’s take a look at some of the high and low first and second derivatives to make sure our definition of derivatives makes sense conceptually.

plotly_data = []

pid = 57

# smoothed individual trajectory
smooth_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
data = smooth_df[['x','y','z']].to_numpy()
tck, u = interpolate.splprep(data.T, k=3)
data = interpolate.splev(np.linspace(0,1,smooth_df['clip_len'].iloc[0]), tck, der=0)
smooth_df['x'] = data[0]
smooth_df['y'] = data[1]
smooth_df['z'] = data[2]
smooth_df[['x_der','y_der','z_der']] = smooth_df['time'].transform(lambda x: smooth_df[['x','y','z']].iloc[x]-smooth_df[['x','y','z']].iloc[x-1])
smooth_df['der'] = smooth_df['time'].transform(lambda x: sqrt(smooth_df['x_der'].iloc[x]**2 + smooth_df['y_der'].iloc[x]**2 + smooth_df['z_der'].iloc[x]**2))
smooth_df[['x_derr','y_derr','z_derr']] = smooth_df['time'].transform(lambda x: smooth_df[['x','y','z']].iloc[x]-2*smooth_df[['x','y','z']].iloc[x-1]+smooth_df[['x','y','z']].iloc[x-2])
smooth_df['derr'] = smooth_df['time'].transform(lambda x: sqrt(smooth_df['x_derr'].iloc[x]**2 + smooth_df['y_derr'].iloc[x]**2 + smooth_df['z_derr'].iloc[x]**2))
smooth_df['clip_len'] = smooth_df['clip_len']-1

pid_traj = go.Scatter3d(
    x=smooth_df['x'],
    y=smooth_df['y'],
    z=smooth_df['z'],
    customdata=smooth_df[['time','clip_len','pid']],
    mode='markers+lines',
    line={'width':4, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    opacity=0.5,
    name='Smoothed',
    legendgroup='Smoothed',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
plotly_data.append(pid_traj)

# high first derivative
temp_df = smooth_df[(smooth_df.time.isin([102,22,7]))]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len','pid','der','derr']],
    mode='markers',
    marker={'size':8, 'color':'darkgreen'},
    opacity=0.8,
    name='High 1st Der',
    legendgroup='High 1st Der',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}<br>1st der: %{customdata[3]:.3f}<br>2nd der: %{customdata[4]:.3f}')
plotly_data.append(pid_traj)
    
# low first derivative
temp_df = smooth_df[(smooth_df.time.isin([218,106,15]))]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len','pid','der','derr']],
    mode='markers',
    marker={'size':8, 'color':'cornflowerblue'},
    opacity=0.8,
    name='Low 1st Der',
    legendgroup='Low 1st Der',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}<br>1st der: %{customdata[3]:.3f}<br>2nd der: %{customdata[4]:.3f}')
plotly_data.append(pid_traj)

# high second derivative
temp_df = smooth_df[(smooth_df.time.isin([89,39,179]))]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len','pid','der','derr']],
    mode='markers',
    marker={'size':8, 'color':'darkred'},
    opacity=0.8,
    name='High 2nd Der',
    legendgroup='High 2nd Der',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}<br>1st der: %{customdata[3]:.3f}<br>2nd der: %{customdata[4]:.3f}')
plotly_data.append(pid_traj)

# low second derivative
temp_df = smooth_df[(smooth_df.time.isin([55,146,233]))]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len','pid','der','derr']],
    mode='markers',
    marker={'size':8, 'color':'darkorange'},
    opacity=0.8,
    name='Low 2nd Der',
    legendgroup='Low 2nd Der',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}<br>1st der: %{customdata[3]:.3f}<br>2nd der: %{customdata[4]:.3f}')
plotly_data.append(pid_traj)

# legend
for color, name in zip(['darkgreen','cornflowerblue','darkred','darkorange'],['High 1st Der','Low 1st Der','High 2nd Der','Low 2nd Der']):
    pid_traj = go.Scatter3d(
        x=[None], y=[None], z=[None],
        mode='markers',
        marker={'size':1, 'color':color},
        opacity=0.8,
        name=name,
        legendgroup=name,
        showlegend=True,)
    plotly_data.append(pid_traj)

# formatting
plotly_layout = go.Layout(
    autosize=False,
    width=800, 
    height=600,
    margin={'l':0, 'r':0, 't':35, 'b':60},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'y':-0.065,
            'x':0.5},
    title={'text':'First and Second Derivative Extrema',
            'xanchor':'center',
            'yanchor':'top',
            'x':0.5,
            'y':0.98},
    annotations=[{'xref':'paper',
                  'yref':'paper',
                  'xanchor':'center',
                  'yanchor':'bottom',
                  'x':0.5,
                  'y':-0.12,
                  'showarrow':False,
                  'text':'<b>Fig. 8.</b> Trajectory for "oceans" participant 57. Some points with high/low first/second derivatives are highlighted.'}])

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation','hoverClosest3d']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

It seems that a high second derivative indicates a curve/bend in the trajectory, while a low second derivative suggest fairly linear motion. This is consistent with the definition of a second derivative.

We can also see similar behavior in the first derivative, where a low first derivative generally suggests a sharp change in direction while a high first derivative indicates linear motion. Although this isn’t how the first derivative is defined for, we can notice that the trajectory slows down right when making sharp turns and maintains a fairly constant velocity when traveling in relatively straight paths. Due to the nature of these trajectories, the first derivative also seems to be a good measure of “bending”.

Now let’s look at these derivatives across all participants watching a single clip. This will hopefully let us find some information about the clip itself, rather than just one participant. We also want to plot the standard deviation as an area round the mean in order to show the variation in the data.

# fig_traj = make_subplots(rows=2, cols=1, 
#                          shared_xaxes=True,
#                          vertical_spacing=0.05,
#                          subplot_titles=('Mean First Derivative', 'Mean Second Derivative'), 
#                          specs=[[{'type':'scatter'}], [{'type':'scatter'}]])

# # def toggle_sd(toggle):
# #     visible = []
# #     for i in range(len(fig_traj['data'])):
# #         visible.append(str(fig_traj['data'][i].visible))
# #     print(visible)
# #     for i in range(0,len(fig_traj['data']),2):
# #         if(visible[i]=='True'):
# #             if(toggle=='show'):
# #                 visible[i+1] = True
# #             elif(toggle=='hide'):
# #                 visible[i+1] = 'legendonly'
# #     return visible

# def show_sd():
#     visible = []
#     for i in range(len(fig_traj['data'])):
#         visible.append(str(fig_traj['data'][i].visible))
#     print(visible)
#     #print(fig_traj['layout'].hiddenlabels)
#     for i in range(0,len(fig_traj['data']),2):
#         if(visible[i]=='True'):
#             visible[i] = True
#             visible[i+1] = True
#     return visible

# def hide_sd():
#     visible = []
#     for i in range(len(fig_traj['data'])):
#         visible.append(str(fig_traj['data'][i].visible))
#     print(visible)
#     #print(fig_traj['layout'].hiddenlabels)
#     for i in range(0,len(fig_traj['data']),2):
#         if(visible[i]=='True'):
#             visible[i] = True
#             visible[i+1] = 'legendonly'
#     return visible

# for clip, clip_name in enumerate(clipdata_df['clip_name']):

#     # smoothed (splines)
#     temp_df = traj_df[(traj_df.clip_name==clip_name)]
#     x=np.zeros(0)
#     y=np.zeros(0)
#     z=np.zeros(0)
#     for pid in range(max(temp_df.pid)):
#         data = temp_df[temp_df.pid==pid+1][['x','y','z']].to_numpy()
#         tck, u = interpolate.splprep(data.T, k=3)
#         data = interpolate.splev(np.linspace(0,1,temp_df['clip_len'].iloc[0]), tck, der=0)
#         x = np.append(x,data[0])
#         y = np.append(y,data[1])
#         z = np.append(z,data[2])
#     temp_df['x'] = x
#     temp_df['y'] = y
#     temp_df['z'] = z

#     temp_df[['x_der','y_der','z_der']] = temp_df[['x','y','z']]-temp_df[['x','y','z']].shift(1).fillna(0)
#     temp_df['der'] = np.sqrt((temp_df[['x_der','y_der','z_der']]**2).sum(axis=1))
#     temp_df[['x_derr','y_derr','z_derr']] = temp_df[['x_der','y_der','z_der']]-temp_df[['x_der','y_der','z_der']].shift(1).fillna(0)
#     temp_df['derr'] = np.sqrt((temp_df[['x_derr','y_derr','z_derr']]**2).sum(axis=1))
#     temp_df['mean_der'] = temp_df.groupby('time')['der'].transform('mean')
#     temp_df['std_der'] = temp_df.groupby('time')['der'].transform('std')
#     temp_df['mean_derr'] = temp_df.groupby('time')['derr'].transform('mean')
#     temp_df['std_derr'] = temp_df.groupby('time')['derr'].transform('std')
#     temp_df = temp_df[~temp_df.time.isin([0,1])]

#     temp_df = temp_df[temp_df.pid==1]
#     temp_df['clip_len'] = temp_df['clip_len']-1
    
#     visibility = 'legendonly'
#     if (temp_df['clip_name'].iloc[0]=='oceans'):
#         visibility = True

#     # first derivative
#     mean_traj = go.Scatter(
#         x=temp_df['time'],
#         y=temp_df['mean_der'],
#         customdata=temp_df[['clip_len','std_der']],
#         mode='markers+lines',
#         line={'width':2, 'color':colors[clip]},
#         marker={'size':4, 'color':colors[clip]},
#         name=clip_name,
#         legendgroup=clip_name,
#         showlegend=True,
#         visible=visibility,
#         hovertemplate='1st der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
#     )
#     fig_traj.add_trace(mean_traj, row=1, col=1)
    
#     upper = temp_df['mean_der'] + temp_df['std_der']
#     lower = temp_df['mean_der'] - temp_df['std_der']
#     std_traj = go.Scatter(
#         x=np.concatenate([temp_df.index, temp_df.index[::-1]])-temp_df.index[0]+2,
#         y=pd.concat([upper, lower[::-1]]),
#         fill='toself',
#         mode='lines',
#         line={'width':0, 'color':colors[clip]},
#         opacity=0.7,
#         name=clip_name,
#         legendgroup=clip_name,
#         showlegend=False,
#         visible=visibility,
#         hoverinfo='skip'
#     )
#     fig_traj.add_trace(std_traj, row=1, col=1)

#     # second derivative
#     mean_traj = go.Scatter(
#         x=temp_df['time'],
#         y=temp_df['mean_derr'],
#         customdata=temp_df[['clip_len','std_derr']],
#         mode='markers+lines',
#         line={'width':2, 'color':colors[clip]},
#         marker={'size':4, 'color':colors[clip]},
#         name=clip_name,
#         legendgroup=clip_name,
#         showlegend=False,
#         visible=visibility,
#         hovertemplate='2nd der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
#     )
#     fig_traj.add_trace(mean_traj, row=2, col=1)
    
#     upper = temp_df['mean_derr'] + temp_df['std_derr']
#     lower = temp_df['mean_derr'] - temp_df['std_derr']
#     std_traj = go.Scatter(
#         x=np.concatenate([temp_df.index, temp_df.index[::-1]])-temp_df.index[0]+2,
#         y=pd.concat([upper, lower[::-1]]),
#         fill='toself',
#         mode='lines',
#         line={'width':0, 'color':colors[clip]},
#         opacity=0.7,
#         name=clip_name,
#         legendgroup=clip_name,
#         showlegend=False,
#         visible=visibility,
#         hoverinfo='skip'
#     )
#     fig_traj.add_trace(std_traj, row=2, col=1)

# # formatting
# fig_traj.update_layout(
#     autosize=False,
#     showlegend=True,
#     width=800, 
#     height=1000, 
#     margin={'l':0, 'r':0, 't':70, 'b':110},
#     legend={'orientation':'h',
#             'itemsizing':'constant',
#             'xanchor':'center',
#             'yanchor':'bottom',
#             'x':0.5,
#             'y':-0.085,
#             'tracegroupgap':2},
#     title={'text':'Mean Individual Trajectory Derivatives',
#             'xanchor':'center',
#             'yanchor':'top',
#             'x':0.5,
#             'y':0.98},
#     hovermode='x',
#     updatemenus=[{'type':'buttons',
#                   'direction':'left',
#                   'pad':{'l':0, 'r':0, 't':0, 'b':0},
#                   'xanchor':'left',
#                   'yanchor':'top',
#                   'x':0,
#                   'y':1.07,
#                   'buttons':[
#                       {'label':'Show SD',
#                        'method': 'update',
#                        'args':[{'visible': show_sd()}]},
#                       {'label':'Hide SD',
#                        'method': 'update',
#                        'args':[{'visible': hide_sd()}]}
#                   ]}])
# fig_traj['layout']['annotations'] += (
#     {'xref':'paper',
#      'yref':'paper',
#      'xanchor':'center',
#      'yanchor':'bottom',
#      'x':0.5,
#      'y':-0.14,
#      'showarrow':False,
#      'text':'<b>Fig. 9.</b> Mean first and second derivatives across all participants for each clip.<br>Error bars show the standard deviation of the mean derivatives at each time point.'
#     },
# )

# plotly_config = {'displaylogo':False,
#                  'modeBarButtonsToRemove': ['autoScale2d','toggleSpikelines','hoverClosestCartesian','hoverCompareCartesian','lasso2d','select2d']}

# fig_traj.show(config=plotly_config)

# print()
# for i in range(10):
#     visible = []
#     for i in range(len(fig_traj['data'])):
#         visible.append(str(fig_traj['data'][i].visible))
#     print(i)
#     print(visible)
#     print()
#     #print(fig_traj['layout'].hiddenlabels)
#     time.sleep(5)

fig_traj = make_subplots(rows=2, cols=2, 
                         shared_xaxes=True,
                         vertical_spacing=0.02, horizontal_spacing=0.05,
                         subplot_titles=('Mean First Derivative','Mean Second Derivative'), 
                         specs=[[{'type':'scatter'}, {'type':'scatter'}], [{'type':'scatter'}, {'type':'scatter'}]])

for clip, clip_name in enumerate(clipdata_df['clip_name']):

    # smoothed (splines)
    temp_df = traj_df[(traj_df.clip_name==clip_name)]
    x=np.zeros(0)
    y=np.zeros(0)
    z=np.zeros(0)
    for pid in range(max(temp_df.pid)):
        data = temp_df[temp_df.pid==pid+1][['x','y','z']].to_numpy()
        tck, u = interpolate.splprep(data.T, k=3)
        data = interpolate.splev(np.linspace(0,1,temp_df['clip_len'].iloc[0]), tck, der=0)
        x = np.append(x,data[0])
        y = np.append(y,data[1])
        z = np.append(z,data[2])
    temp_df['x'] = x
    temp_df['y'] = y
    temp_df['z'] = z

    temp_df[['x_der','y_der','z_der']] = temp_df[['x','y','z']]-temp_df[['x','y','z']].shift(1).fillna(0)
    temp_df['der'] = np.sqrt((temp_df[['x_der','y_der','z_der']]**2).sum(axis=1))
    temp_df[['x_derr','y_derr','z_derr']] = temp_df[['x_der','y_der','z_der']]-temp_df[['x_der','y_der','z_der']].shift(1).fillna(0)
    temp_df['derr'] = np.sqrt((temp_df[['x_derr','y_derr','z_derr']]**2).sum(axis=1))
    temp_df['mean_der'] = temp_df.groupby('time')['der'].transform('mean')
    temp_df['std_der'] = temp_df.groupby('time')['der'].transform('std')
    temp_df['mean_derr'] = temp_df.groupby('time')['derr'].transform('mean')
    temp_df['std_derr'] = temp_df.groupby('time')['derr'].transform('std')
    
    temp_df = temp_df[~temp_df.time.isin([0,1])]
    temp_df = temp_df[temp_df.pid==1]
    temp_df['clip_len'] = temp_df['clip_len']-1
    
    visibility = 'legendonly'
    if (clip_name=='oceans'):
        visibility = True

    # first derivative (no std)
    mean_traj = go.Scatter(
        x=temp_df['time'],
        y=temp_df['mean_der'],
        customdata=temp_df[['clip_len','std_der']],
        mode='markers+lines',
        line={'width':2, 'color':colors[clip]},
        marker={'size':4, 'color':colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        showlegend=True,
        visible=visibility,
        hovertemplate='1st der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
    )
    fig_traj.add_trace(mean_traj, row=1, col=1)
    
    # first derivative (std)
    mean_traj = go.Scatter(
        x=temp_df['time'],
        y=temp_df['mean_der'],
        customdata=temp_df[['clip_len','std_der']],
        mode='markers+lines',
        line={'width':2, 'color':colors[clip]},
        marker={'size':4, 'color':colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hovertemplate='1st der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
    )
    fig_traj.add_trace(mean_traj, row=2, col=1)
    
    upper = temp_df['mean_der'] + temp_df['std_der']
    lower = temp_df['mean_der'] - temp_df['std_der']
    std_traj = go.Scatter(
        x=np.concatenate([temp_df.index, temp_df.index[::-1]])-temp_df.index[0]+2,
        y=pd.concat([upper, lower[::-1]]),
        fill='toself',
        mode='lines',
        line={'width':0, 'color':colors[clip]},
        opacity=0.7,
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hoverinfo='skip'
    )
    fig_traj.add_trace(std_traj, row=2, col=1)

    # second derivative (no std)
    mean_traj = go.Scatter(
        x=temp_df['time'],
        y=temp_df['mean_derr'],
        customdata=temp_df[['clip_len','std_derr']],
        mode='markers+lines',
        line={'width':2, 'color':colors[clip]},
        marker={'size':4, 'color':colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hovertemplate='2nd der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
    )
    fig_traj.add_trace(mean_traj, row=1, col=2)
    
    # second derivative (std)
    mean_traj = go.Scatter(
        x=temp_df['time'],
        y=temp_df['mean_derr'],
        customdata=temp_df[['clip_len','std_derr']],
        mode='markers+lines',
        line={'width':2, 'color':colors[clip]},
        marker={'size':4, 'color':colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hovertemplate='2nd der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
    )
    fig_traj.add_trace(mean_traj, row=2, col=2)
    
    upper = temp_df['mean_derr'] + temp_df['std_derr']
    lower = temp_df['mean_derr'] - temp_df['std_derr']
    std_traj = go.Scatter(
        x=np.concatenate([temp_df.index, temp_df.index[::-1]])-temp_df.index[0]+2,
        y=pd.concat([upper, lower[::-1]]),
        fill='toself',
        mode='lines',
        line={'width':0, 'color':colors[clip]},
        opacity=0.7,
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hoverinfo='skip'
    )
    fig_traj.add_trace(std_traj, row=2, col=2)

# formatting
fig_traj.update_layout(
    autosize=False,
    showlegend=True,
    width=1050, 
    height=900, 
    margin={'l':0, 'r':0, 't':70, 'b':110},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'x':0.5,
            'y':-0.09,
            'tracegroupgap':2},
    title={'text':'Mean Individual Trajectory Derivatives',
            'xanchor':'center',
            'yanchor':'top',
            'x':0.5,
            'y':0.98},
    hovermode='x')
fig_traj['layout']['annotations'] += (
    {'xref':'paper',
     'yref':'paper',
     'xanchor':'center',
     'yanchor':'bottom',
     'x':0.5,
     'y':-0.155,
     'showarrow':False,
     'text':'<b>Fig. 9.</b> Mean first and second derivatives across all participants for each clip.<br>Error bars show the standard deviation of the mean derivatives at each time point.'
    },
)

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['autoScale2d','toggleSpikelines','hoverClosestCartesian','hoverCompareCartesian','lasso2d','select2d']}

fig_traj.show(config=plotly_config)

From these plots, we can see that the mean first derivative has a general downward trend across time for most clips. This makes sense since the trajectory could slow down as it reaches the trajectory end, where it remains in a relatively small area for many time points.

From looking at individual and mean trajectories, we see that there is a lot of noise in the trajectory end. The trajectory bends a lot due to fluctuations as it stays in the end area. However, the second derivatives don’t seem to correspond with this idea, with most clips having either no trend or a slight upward trend across time for most clips. This may be due to some individual trajectories that bend a lot even at the start.

Clip length seems to be a significant factor as well. First derivatives appear to decrease across time at a similar rate for all clips. This leaves shorter clips like “overcome”, “testretest”, and “dreary” with relatively high first derivatives at the clip end, while longer clips such as “starwars”, “oceans”, and “twomen” end with relatively low first derivatives. Similarly, second derivatives seem to increase at a faster rate for shorter clips and remain more constant for longer clips.

It should be noted that the derivatives increase significantly at the last two to three time steps for each clip. I’m currently unsure why this may be ocurring, but it may have to do with how participants react to an ending clip, which may be sudden in many cases.

fig_traj = make_subplots(rows=2, cols=2, 
                         shared_xaxes=True,
                         vertical_spacing=0.02, horizontal_spacing=0.05,
                         subplot_titles=('Mean First Derivative','Mean Second Derivative'), 
                         specs=[[{'type':'scatter'}, {'type':'scatter'}], [{'type':'scatter'}, {'type':'scatter'}]])

for clip, clip_name in enumerate(clipdata_df['clip_name']):

    # smoothed (splines)
    temp_df = traj_df[(traj_df.clip_name==clip_name)]
    num_pids = max(temp_df.pid)
    num_points = (temp_df['clip_len'].iloc[0]-1)*10+1
    clip_id = [temp_df['clip'].iloc[0]]*num_points*num_pids
    clip_names = [temp_df['clip_name'].iloc[0]]*num_points*num_pids
    clip_len = [temp_df['clip_len'].iloc[0]-1]*num_points*num_pids
    x_der = np.zeros(0)
    y_der = np.zeros(0)
    z_der = np.zeros(0)
    x_derr = np.zeros(0)
    y_derr = np.zeros(0)
    z_derr = np.zeros(0)
    pids = np.zeros(0)
    time = np.zeros(0)
    for curr_pid in range(num_pids):
        data = temp_df[temp_df.pid==curr_pid+1][['x','y','z']].to_numpy()
        tck, u = interpolate.splprep(data.T, k=3)
        der = interpolate.splev(np.linspace(0,1,num_points), tck, der=1)
        derr = interpolate.splev(np.linspace(0,1,num_points), tck, der=2)
        x_der = np.append(x_der,der[0])
        y_der = np.append(y_der,der[1])
        z_der = np.append(z_der,der[2])
        x_derr = np.append(x_derr,derr[0])
        y_derr = np.append(y_derr,derr[1])
        z_derr = np.append(z_derr,derr[2])
        pids = np.append(pids,[curr_pid]*num_points)
        time = np.append(time,np.linspace(0,1,num_points)*(temp_df['clip_len'].iloc[0]-1))
        
    temp_df = pd.DataFrame({'clip':clip_id, 'clip_name':clip_names, 'clip_len':clip_len, 'pid':pids, 'time':time,
                            'x_der':x_der, 'y_der':y_der, 'z_der':z_der,
                            'x_derr':x_derr, 'y_derr':y_derr, 'z_derr':z_derr},
                           columns=['clip','clip_name','clip_len','pid','time','x_der','y_der','z_der','x_derr','y_derr','z_derr'])

    temp_df['der'] = np.sqrt((temp_df[['x_der','y_der','z_der']]**2).sum(axis=1))
    temp_df['derr'] = np.sqrt((temp_df[['x_derr','y_derr','z_derr']]**2).sum(axis=1))
    temp_df['mean_der'] = temp_df.groupby('time')['der'].transform('mean')
    temp_df['std_der'] = temp_df.groupby('time')['der'].transform('std')
    temp_df['mean_derr'] = temp_df.groupby('time')['derr'].transform('mean')
    temp_df['std_derr'] = temp_df.groupby('time')['derr'].transform('std')
    
    temp_df = temp_df[temp_df.pid==1]
    
    visibility = 'legendonly'
    if (clip_name=='oceans'):
        visibility = True

    # first derivative (no std)
    mean_traj = go.Scatter(
        x=temp_df['time'],
        y=temp_df['mean_der'],
        customdata=temp_df[['clip_len','std_der']],
        mode='markers+lines',
        line={'width':2, 'color':colors[clip]},
        marker={'size':4, 'color':colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        showlegend=True,
        visible=visibility,
        hovertemplate='1st der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
    )
    fig_traj.add_trace(mean_traj, row=1, col=1)
    
    # first derivative (std)
    mean_traj = go.Scatter(
        x=temp_df['time'],
        y=temp_df['mean_der'],
        customdata=temp_df[['clip_len','std_der']],
        mode='markers+lines',
        line={'width':2, 'color':colors[clip]},
        marker={'size':4, 'color':colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hovertemplate='1st der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
    )
    fig_traj.add_trace(mean_traj, row=2, col=1)
    
    upper = temp_df['mean_der'] + temp_df['std_der']
    lower = temp_df['mean_der'] - temp_df['std_der']
    std_traj = go.Scatter(
        x=(np.concatenate([temp_df.index, temp_df.index[::-1]])-temp_df.index[0])/10,
        y=pd.concat([upper, lower[::-1]]),
        fill='toself',
        mode='lines',
        line={'width':0, 'color':colors[clip]},
        opacity=0.7,
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hoverinfo='skip'
    )
    fig_traj.add_trace(std_traj, row=2, col=1)

    # second derivative (no std)
    mean_traj = go.Scatter(
        x=temp_df['time'],
        y=temp_df['mean_derr'],
        customdata=temp_df[['clip_len','std_derr']],
        mode='markers+lines',
        line={'width':2, 'color':colors[clip]},
        marker={'size':4, 'color':colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hovertemplate='2nd der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
    )
    fig_traj.add_trace(mean_traj, row=1, col=2)
    
    # second derivative (std)
    mean_traj = go.Scatter(
        x=temp_df['time'],
        y=temp_df['mean_derr'],
        customdata=temp_df[['clip_len','std_derr']],
        mode='markers+lines',
        line={'width':2, 'color':colors[clip]},
        marker={'size':4, 'color':colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hovertemplate='2nd der: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
    )
    fig_traj.add_trace(mean_traj, row=2, col=2)
    
    upper = temp_df['mean_derr'] + temp_df['std_derr']
    lower = temp_df['mean_derr'] - temp_df['std_derr']
    std_traj = go.Scatter(
        x=(np.concatenate([temp_df.index, temp_df.index[::-1]])-temp_df.index[0])/10,
        y=pd.concat([upper, lower[::-1]]),
        fill='toself',
        mode='lines',
        line={'width':0, 'color':colors[clip]},
        opacity=0.7,
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hoverinfo='skip'
    )
    fig_traj.add_trace(std_traj, row=2, col=2)

# formatting
fig_traj.update_layout(
    autosize=False,
    showlegend=True,
    width=1050, 
    height=900, 
    margin={'l':0, 'r':0, 't':70, 'b':110},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'x':0.5,
            'y':-0.09,
            'tracegroupgap':2},
    title={'text':'Mean Individual Trajectory Derivatives',
            'xanchor':'center',
            'yanchor':'top',
            'x':0.5,
            'y':0.98},
    hovermode='x')
fig_traj['layout']['annotations'] += (
    {'xref':'paper',
     'yref':'paper',
     'xanchor':'center',
     'yanchor':'bottom',
     'x':0.5,
     'y':-0.155,
     'showarrow':False,
     'text':'<b>Fig. 9.</b> Mean first and second derivatives across all participants for each clip.<br>Error bars show the standard deviation of the mean derivatives at each time point.'
    },
)

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['autoScale2d','toggleSpikelines','hoverClosestCartesian','hoverCompareCartesian','lasso2d','select2d']}

fig_traj.show(config=plotly_config)

Curvature

Another way to measure “bending” is with curvature, where higher curvature indicates more bending. Curvature is defined as

\[k=\dfrac{\rvert\rvert\ \mathbf{r}'(t)\ \times\ \mathbf{r}''(t)\ \rvert\rvert}{\rvert\rvert\ \mathbf{r}'(t)\ \rvert\rvert ^3}\]

Since curvature uses the first and second derivatives, we’ll simply use the same definitions as before.

fig_traj = make_subplots(rows=2, cols=1, 
                         shared_xaxes=True,
                         vertical_spacing=0.07,
                         subplot_titles=('Unsmoothed', 'Smoothed'), 
                         specs=[[{'type':'scatter'}], [{'type':'scatter'}]])

pid = 57
    
# unsmoothed
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
temp_df['clip_len'] = temp_df['clip_len']-1
temp_df[['x_der','y_der','z_der']] = temp_df['time'].transform(lambda x: temp_df[['x','y','z']].iloc[x]-temp_df[['x','y','z']].iloc[x-1])
temp_df[['x_derr','y_derr','z_derr']] = temp_df['time'].transform(lambda x: temp_df[['x_der','y_der','z_der']].iloc[x]-temp_df[['x_der','y_der','z_der']].iloc[x-1])
temp_df['k'] = temp_df['time'].transform(lambda x: np.linalg.norm(np.cross(temp_df[['x_der','y_der','z_der']].iloc[x].to_numpy(), temp_df[['x_derr','y_derr','z_derr']].iloc[x].to_numpy())) / np.linalg.norm(temp_df[['x_der','y_der','z_der']].iloc[x].to_numpy())**3)
temp_df = temp_df.iloc[ -(temp_df['clip_len'].iloc[0]-1): ]
pid_traj = go.Scatter(
    x=temp_df['time'],
    y=temp_df['k'],
    customdata=temp_df[['clip_len','pid']],
    mode='markers+lines',
    line={'width':2, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    name='Unsmoothed',
    legendgroup='Unsmoothed',
    showlegend=False,
    hovertemplate='k: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>pid: %{customdata[1]}')
fig_traj.add_trace(pid_traj, row=1, col=1)

# smoothed (splines)
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clipdata_df['clip_len'].iloc[x]-1)
data = temp_df[['x','y','z']].to_numpy()
tck, u = interpolate.splprep(data.T, k=3)
data = interpolate.splev(np.linspace(0,1,temp_df['clip_len'].iloc[0]+1), tck, der=0)
temp_df['x'] = data[0]
temp_df['y'] = data[1]
temp_df['z'] = data[2]
temp_df[['x_der','y_der','z_der']] = temp_df['time'].transform(lambda x: temp_df[['x','y','z']].iloc[x]-temp_df[['x','y','z']].iloc[x-1])
temp_df[['x_derr','y_derr','z_derr']] = temp_df['time'].transform(lambda x: temp_df[['x_der','y_der','z_der']].iloc[x]-temp_df[['x_der','y_der','z_der']].iloc[x-1])
temp_df['k_1'] = temp_df['time'].transform(lambda x: np.linalg.norm(np.cross(temp_df[['x_der','y_der','z_der']].iloc[x].to_numpy(), temp_df[['x_derr','y_derr','z_derr']].iloc[x].to_numpy())))
temp_df['k_2'] = temp_df['time'].transform(lambda x: np.linalg.norm(temp_df[['x_der','y_der','z_der']].iloc[x].to_numpy())**3)
temp_df['k'] = temp_df['time'].transform(lambda x: np.linalg.norm(np.cross(temp_df[['x_der','y_der','z_der']].iloc[x].to_numpy(), temp_df[['x_derr','y_derr','z_derr']].iloc[x].to_numpy())) / np.linalg.norm(temp_df[['x_der','y_der','z_der']].iloc[x].to_numpy())**3)
temp_df = temp_df.iloc[ -(temp_df['clip_len'].iloc[0]-1): ]
pid_traj = go.Scatter(
    x=temp_df['time'],
    y=temp_df['k'],
    customdata=temp_df[['clip_len','pid']],
    line={'width':2, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    name='Smoothed',
    legendgroup='Smoothed',
    showlegend=False,
    hovertemplate='k: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>pid: %{customdata[1]}'
)
fig_traj.add_trace(pid_traj, row=2, col=1)
    
# formatting
fig_traj.update_layout(
    autosize=False,
    width=800, 
    height=800, 
    margin={'l':0, 'r':0, 't':70, 'b':80},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'y':-0.055,
            'x':0.5},
    title={'text':'Individual Trajectory Curvature',
            'xanchor':'center',
            'yanchor':'top',
            'x':0.5,
            'y':0.98},
    hovermode='x')
fig_traj['layout']['annotations'] += (
    {'xref':'paper',
     'yref':'paper',
     'xanchor':'center',
     'yanchor':'bottom',
     'x':0.5,
     'y':-0.12,
     'showarrow':False,
     'text':'<b>Fig. 10.</b> Curvature of individual trajectories for "oceans" participant 57.<br>(A) Original unsmoothed trajectory. (B) Smoothed with cubic splines.'
    },
)

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['autoScale2d','toggleSpikelines','hoverClosestCartesian','hoverCompareCartesian','lasso2d','select2d']}

fig_traj.show(config=plotly_config)

This seems like a promising way to quantify bending since it gives very distinct spikes.

Similar to with derivatives, we want to plot some points with high and low curvature to make sure these concepts apply to trajectories.

plotly_data = []

pid = 57

# smoothed individual trajectory
smooth_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
smooth_df['clip_len'] = smooth_df['clip_len']-1
data = smooth_df[['x','y','z']].to_numpy()
tck, u = interpolate.splprep(data.T, k=3)
data = interpolate.splev(np.linspace(0,1,smooth_df['clip_len'].iloc[0]+1), tck, der=0)
smooth_df['x'] = data[0]
smooth_df['y'] = data[1]
smooth_df['z'] = data[2]
smooth_df[['x_der','y_der','z_der']] = smooth_df['time'].transform(lambda x: smooth_df[['x','y','z']].iloc[x]-smooth_df[['x','y','z']].iloc[x-1])
smooth_df[['x_derr','y_derr','z_derr']] = temp_df['time'].transform(lambda x: smooth_df[['x_der','y_der','z_der']].iloc[x]-smooth_df[['x_der','y_der','z_der']].iloc[x-1])
smooth_df['k'] = smooth_df['time'].transform(lambda x: np.linalg.norm(np.cross(smooth_df[['x_der','y_der','z_der']].iloc[x].to_numpy(), smooth_df[['x_derr','y_derr','z_derr']].iloc[x].to_numpy())) / np.linalg.norm(smooth_df[['x_der','y_der','z_der']].iloc[x].to_numpy())**3)

pid_traj = go.Scatter3d(
    x=smooth_df['x'],
    y=smooth_df['y'],
    z=smooth_df['z'],
    customdata=smooth_df[['time','clip_len','pid']],
    mode='markers+lines',
    line={'width':4, 'color':'mediumblue'},
    marker={'size':1, 'color':'mediumblue'},
    opacity=0.5,
    name='Smoothed',
    legendgroup='Smoothed',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
plotly_data.append(pid_traj)

# high curvature
temp_df = smooth_df[(smooth_df.time.isin([106,148,178,204,218]))]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len','pid','k']],
    mode='markers',
    marker={'size':8, 'color':'darkred'},
    opacity=0.8,
    name='High Curvature',
    legendgroup='High Curvature',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}<br>k: %{customdata[3]:.3f}')
plotly_data.append(pid_traj)

# low curvature
temp_df = smooth_df[(smooth_df.time.isin([8,22,110,201,216]))]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len','pid','k']],
    mode='markers',
    marker={'size':8, 'color':'darkorange'},
    opacity=0.8,
    name='Low Curvature',
    legendgroup='Low Curvature',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}<br>k: %{customdata[3]:.3f}')
plotly_data.append(pid_traj)    

# legend
for color, name in zip(['darkred','darkorange'],['High Curvature','Low Curvature']):
    pid_traj = go.Scatter3d(
        x=[None], y=[None], z=[None],
        mode='markers',
        marker={'size':1, 'color':color},
        opacity=0.8,
        name=name,
        legendgroup=name,
        showlegend=True,)
    plotly_data.append(pid_traj)

# formatting
plotly_layout = go.Layout(
    autosize=False,
    width=800, 
    height=600,
    margin={'l':0, 'r':0, 't':35, 'b':60},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'y':-0.065,
            'x':0.5},
    title={'text':'First and Second Derivative Extrema',
            'xanchor':'center',
            'yanchor':'top',
            'x':0.5,
            'y':0.98},
    annotations=[{'xref':'paper',
                  'yref':'paper',
                  'xanchor':'center',
                  'yanchor':'bottom',
                  'x':0.5,
                  'y':-0.12,
                  'showarrow':False,
                  'text':'<b>Fig. 11.</b> Trajectory for "oceans" participant 57. Some points with high/low curvature are highlighted.'}])

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation','hoverClosest3d']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

These points seem to match up perfectly with what we want curvature to represent. Points with low curvature suggest fairly linear motion and points with high curvature represent a bending trajectory.

Now let’s find curvature across all participants for each clip.

fig_traj = make_subplots(rows=2, cols=1, 
                         shared_xaxes=True,
                         vertical_spacing=0.02, horizontal_spacing=0.05,
                         subplot_titles=('Mean Curvature',''), 
                         specs=[[{'type':'scatter'}],[{'type':'scatter'}]])

for clip, clip_name in enumerate(clipdata_df['clip_name']):

    # smoothed (splines)
    temp_df = traj_df[(traj_df.clip_name==clip_name)]
    x=np.zeros(0)
    y=np.zeros(0)
    z=np.zeros(0)
    for pid in range(max(temp_df.pid)):
        data = temp_df[temp_df.pid==pid+1][['x','y','z']].to_numpy()
        tck, u = interpolate.splprep(data.T, k=3)
        data = interpolate.splev(np.linspace(0,1,temp_df['clip_len'].iloc[0]), tck, der=0)
        x = np.append(x,data[0])
        y = np.append(y,data[1])
        z = np.append(z,data[2])
    temp_df['x'] = x
    temp_df['y'] = y
    temp_df['z'] = z

    temp_df[['x_der','y_der','z_der']] = temp_df[['x','y','z']]-temp_df[['x','y','z']].shift(1).fillna(0)
    temp_df[['x_derr','y_derr','z_derr']] = temp_df[['x_der','y_der','z_der']]-temp_df[['x_der','y_der','z_der']].shift(1).fillna(0)
    temp_df['derr'] = np.sqrt((temp_df[['x_derr','y_derr','z_derr']]**2).sum(axis=1))
    temp_df['k'] = np.linalg.norm(np.cross(temp_df[['x_der','y_der','z_der']].to_numpy(), temp_df[['x_derr','y_derr','z_derr']].to_numpy()), axis=1) / np.linalg.norm(temp_df[['x_der','y_der','z_der']].to_numpy(), axis=1)**3
    
    temp_df['mean_k'] = temp_df.groupby('time')['k'].transform('mean')
    temp_df['std_k'] = temp_df.groupby('time')['k'].transform('std')
    
    temp_df = temp_df[~temp_df.time.isin([0,1])]
    temp_df = temp_df[temp_df.pid==1]
    temp_df['clip_len'] = temp_df['clip_len']-1
    
    visibility = 'legendonly'
    if (clip_name=='oceans'):
        visibility = True

    # curvature (no std)
    mean_traj = go.Scatter(
        x=temp_df['time'],
        y=temp_df['mean_k'],
        customdata=temp_df[['clip_len','std_k']],
        mode='markers+lines',
        line={'width':2, 'color':colors[clip]},
        marker={'size':4, 'color':colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        showlegend=True,
        visible=visibility,
        hovertemplate='k: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
    )
    fig_traj.add_trace(mean_traj, row=1, col=1)
    
    # curvature (std)
    mean_traj = go.Scatter(
        x=temp_df['time'],
        y=temp_df['mean_k'],
        customdata=temp_df[['clip_len','std_k']],
        mode='markers+lines',
        line={'width':2, 'color':colors[clip]},
        marker={'size':4, 'color':colors[clip]},
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hovertemplate='k: %{y:.3f}<br>t: %{x}/%{customdata[0]}<br>sd: %{customdata[1]:.3f}'
    )
    fig_traj.add_trace(mean_traj, row=2, col=1)
    
    upper = temp_df['mean_k'] + temp_df['std_k']
    lower = temp_df['mean_k'] - temp_df['std_k']
    std_traj = go.Scatter(
        x=np.concatenate([temp_df.index, temp_df.index[::-1]])-temp_df.index[0]+2,
        y=pd.concat([upper, lower[::-1]]),
        fill='toself',
        mode='lines',
        line={'width':0, 'color':colors[clip]},
        opacity=0.7,
        name=clip_name,
        legendgroup=clip_name,
        showlegend=False,
        visible=visibility,
        hoverinfo='skip'
    )
    fig_traj.add_trace(std_traj, row=2, col=1)

# formatting
fig_traj.update_layout(
    autosize=False,
    showlegend=True,
    width=800, 
    height=800, 
    margin={'l':0, 'r':0, 't':70, 'b':110},
    legend={'orientation':'h',
            'itemsizing':'constant',
            'xanchor':'center',
            'yanchor':'bottom',
            'x':0.5,
            'y':-0.085,
            'tracegroupgap':2},
    title={'text':'Mean Individual Trajectory Curvature',
            'xanchor':'center',
            'yanchor':'top',
            'x':0.5,
            'y':0.98},
    hovermode='x')
fig_traj['layout']['annotations'] += (
    {'xref':'paper',
     'yref':'paper',
     'xanchor':'center',
     'yanchor':'bottom',
     'x':0.5,
     'y':-0.14,
     'showarrow':False,
     'text':'<b>Fig. 12.</b> Mean curvature across all participants for each clip.<br>Error bars show the standard deviation of the mean curvature at each time point.'
    },
)

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['autoScale2d','toggleSpikelines','hoverClosestCartesian','hoverCompareCartesian','lasso2d','select2d']}

fig_traj.show(config=plotly_config)

This plot presents a problem with using curvature to quantify bending across clips. While curvature worked well for individual plots, all spikes seen in these plots have extremely high standard deviation, suggesting that there are only a few individual trajectories with spikes skewing the data. Therefore, this plot tells us more about individual trajectories rather than the overall clip. Using derivatives to quantify bending seems like a better method.

Trajectory Divergence

Single Trajectory

Total Divergence

# compares adjacent distances
def divergence_v1(traj):
    length = traj.shape[0]
    total_dist = 0
    for i in range(1,length):
        total_dist += sqrt((traj[i,0]-traj[i-1,0])**2 + (traj[i,1]-traj[i-1,1])**2 + (traj[i,2]-traj[i-1,2])**2)
    
    inc = 3 # increment = number of time steps to move forward
    dist = np.zeros(length-inc)
    for i in range(inc,length):
        dist[i-inc] = sqrt((traj[i,0]-traj[i-inc,0])**2 + (traj[i,1]-traj[i-inc,1])**2 + (traj[i,2]-traj[i-inc,2])**2)
    
    total_div = 0
    for start in range(inc,2*inc):
        div = 0
        for i in range(start, length-inc, inc):
            div += abs(dist[i] - dist[i-inc])
        total_div += div
    
    return(total_div / (inc*total_dist))



# compares all distances
def divergence_v2(traj):
    length = traj.shape[0]
    total_dist = 0
    for i in range(1,length):
        total_dist += sqrt((traj[i,0]-traj[i-1,0])**2 + (traj[i,1]-traj[i-1,1])**2 + (traj[i,2]-traj[i-1,2])**2)
    
    inc = 3 # increment = number of time steps to move forward
    dist = np.zeros(length-inc)
    for i in range(inc,length):
        dist[i-inc] = sqrt((traj[i,0]-traj[i-inc,0])**2 + (traj[i,1]-traj[i-inc,1])**2 + (traj[i,2]-traj[i-inc,2])**2)
    
    total_div = 0
    for i in range(len(dist)-1):
        for j in range(i+1,len(dist)):
            total_div += abs(dist[i]-dist[j])
    
    return(total_div / (inc*total_dist))

Now let’s make sure our definition of divergence makes sense for some defined example trajectories.

Work in progress (currently priority #2)

def test_divergence(name):
    if (name=='line_constant'): # expected divergence = 0
        length = random.randint(50,100)
        traj_arr = np.zeros((length,3))
        segment_len = random.randint(0,10)
        direction = np.random.rand(3)
        for i in range(1,length):
            for j in range(len(direction)):
                traj_arr[i,j] = traj_arr[i-1,j] + segment_len*direction[j]
                
    elif (name=='line_random'): # expected divergence = 0
        length = random.randint(50,100)
        traj_arr = np.zeros((length,3))
        direction = np.random.rand(3)
        for i in range(1,length):
            segment_len = random.randint(0,10)
            for j in range(len(direction)):
                traj_arr[i,j] = traj_arr[i-1,j] + segment_len*direction[j]
                
    elif (name=='line_increasing'): # expected divergence = 0
        length = random.randint(50,100)
        traj_arr = np.zeros((length,3))
        direction = np.random.rand(3)
        segment_len = random.randint(0,10)
        for i in range(1,length):
            for j in range(len(direction)):
                traj_arr[i,j] = traj_arr[i-1,j] + segment_len*direction[j]
            segment_len = segment_len * 1.5
        
    elif (name=='semicircle'): # expected divergence = 0
        length = random.randint(50,100)
        traj_arr = np.zeros((length,3))
        radius = random.random()*50
        for i in range(length):
            traj_arr[i,0] = radius * cos(pi*i/length)
            traj_arr[i,1] = radius * sin(pi*i/length)
        
#     elif (name=='curve'):

#     elif (name=='zigzag'):
        
    return divergence_v1(traj_arr)

print(f"Divergence of line with constant velocity: {test_divergence('line_constant'):.3f}")
print(f"Divergence of line with random velocity: {test_divergence('line_random'):.3f}")
print(f"Divergence of line with increasing velocity: {test_divergence('line_increasing'):.3f}")
print(f"Divergence of semicicle: {test_divergence('semicircle'):.3f}")

Divergence of line with constant velocity: 0.000
Divergence of line with random velocity: 0.388
Divergence of line with increasing velocity: 0.495
Divergence of semicicle: 0.000

temp_df = traj_df[(traj_df.clip_name=='overcome') & (traj_df.pid==1)]
single_traj = np.array(temp_df[['x','y','z']])
print(f"Divergence of single trajectory: {divergence_v1(single_traj):.3f}")

mean_traj = np.array(temp_df[['mean_x','mean_y','mean_z']])
print(f"Divergence of mean trajectory: {divergence_v1(mean_traj):.3f}")

Divergence of single trajectory: 0.365
Divergence of mean trajectory: 0.290

Divergence as a Function of Time

Work in progress (currently priority #1)

Need to define a function and create a figure to show how it works. Then I can start testing on defined examples.

Between Two Trajectories

Work in progress (currently priority #3)

Need to define a function and create a figure to show how it works.

IMDb with LSTMs